Image processing system, image processing method, and storage medium

ABSTRACT

An example embodiment includes: an extraction unit that extracts a determination object image including a side part of an outer circumference of an iris from an image including an eye; and a determination unit that determines whether or not a colored contact lens is worn based on the determination object image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/485,594 entitled “IMAGE PROCESSING SYSTEM, IMAGE PROCESSING METHOD,AND STORAGE MEDIUM,” filed on Aug. 13, 2019, which is a national stageapplication of International Application No. PCT/JP2018/032052 entitled“Image-Processing System, Image-Processing Method, and Recording Medium”filed on Aug. 30, 2018, which claims priority to Japanese PatentApplication No. 2017-165404 filed on Aug. 30, 2017, the disclosures ofwhich are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

The present invention relates to an image processing system, an imageprocessing method, and a storage medium.

BACKGROUND ART

Patent Literature 1 discloses an individual authentication method usingan iris image. In such an individual authentication method, a maskregion is set under an upper eyelid, an image of an iris regionexcluding the mask region is used for authentication, and therebyeyelashes less affect the authentication accuracy.

CITATION LIST Patent Literature

-   [PTL 1] Japanese Patent Application Laid-Open No. 2004-206444

SUMMARY OF INVENTION Technical Problem

In recent years, colored contact lenses in which a colored region isprovided to a part of the contact lens have been commercially available.A colored contact lens on an eye provides a decorative effect thatchanges the color of an iris or makes an iris appear larger, and thuscolored contact lenses have spread through the market.

In recognition of an iris as disclosed in Patent Literature 1, when therecognition subject wears colored contact lenses, appearance of a partof an iris changes. Thus, in order to perform appropriate matching, itis necessary to detect whether or not the colored contact lenses areworn.

The present invention has been made in view of the above problem andintends to provide an image processing system, an image processingmethod, and a storage medium that can accurately determine whether ornot a recognition subject wears colored contact lenses.

Solution to Problem

According to one example aspect of the present invention, provided is animage processing system including: an extraction unit that extracts adetermination object image including a side part of an outercircumference of an iris from an image including an eye; and adetermination unit that determines whether or not a colored contact lensis worn based on the determination object image.

According to another example aspect of the present invention, providedis an image processing method including: extracting a determinationobject image including a side part of an outer circumference of an irisfrom an image including an eye; and determining whether or not a coloredcontact lens is worn based on the determination object image.

According to yet another example aspect of the present invention,provided is a storage medium in which a program is stored, the programcausing a computer to execute: extracting a determination object imageincluding a side part of an outer circumference of an iris from an imageincluding an eye; and determining whether or not a colored contact lensis worn based on the determination object image.

Advantageous Effects of Invention

According to the present invention, an image processing system, an imageprocessing method, and a storage medium that can accurately determinewhether or not a recognition subject wears colored contact lenses can beprovided.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a hardware configuration exampleof an iris recognition system according to a first example embodiment.

FIG. 2 is a function block diagram of the iris recognition systemaccording to the first example embodiment.

FIG. 3 is a flowchart illustrating a summary of a process performed bythe iris recognition system according to the first example embodiment.

FIG. 4 is a flowchart illustrating a colored contact lens-wearingdetermination process according to the first example embodiment.

FIG. 5 is a diagram schematically illustrating extraction ofdetermination object images according to the first example embodiment.

FIG. 6 is a diagram schematically illustrating change of the size of adetermination object image according to the first example embodiment.

FIG. 7 is a diagram schematically illustrating a determination processaccording to the first example embodiment.

FIG. 8 is a flowchart illustrating a first example of a matchingaccuracy improvement process according to the first example embodiment.

FIG. 9 is a flowchart illustrating a second example of a matchingaccuracy improvement process according to the first example embodiment.

FIG. 10 is a flowchart illustrating a third example of a matchingaccuracy improvement process according to the first example embodiment.

FIG. 11 is a flowchart illustrating a modified example of a thirdexample of a matching accuracy improvement process according to thefirst example embodiment.

FIG. 12 is a flowchart illustrating an iris matching process accordingto the first example embodiment.

FIG. 13 is a diagram schematically illustrating a setting of anextraction range of a feature amount according to the first exampleembodiment.

FIG. 14 is a function block diagram of an iris recognition systemaccording to a second example embodiment.

FIG. 15 is a function block diagram of an image processing systemaccording to a third example embodiment.

FIG. 16 is a function block diagram of an iris recognition systemaccording to a fourth example embodiment.

DESCRIPTION OF EMBODIMENTS

Example embodiments of the present invention will be described belowwith reference to the drawings. Throughout the drawings, the sameelements or corresponding elements are labeled with the same referencenumerals, and the description thereof may be omitted or simplified.

First Example Embodiment

FIG. 1 is a block diagram illustrating a hardware configuration exampleof an iris recognition system 10 that performs matching of an irisaccording to a first example embodiment of the present invention. Theiris recognition system 10 may be a computer or an informationcommunication terminal such as a mobile phone, a smartphone, a tabletpersonal computer (PC), a laptop PC, a desktop PC, or the like, forexample. The iris recognition system 10 has a function of iris matchingthat is a sort of biometric matching. The iris recognition system 10performs iris matching by capturing an image of an iris of a user thatis a recognition subject and matching it against a registered irisimage. The pattern of an iris that adjust an aperture of a pupil isunique and permanent. Thus, identity verification is possible bymatching the iris pattern acquired at matching against an iris imagethat has been registered in advance. In the following description, it isassumed that the iris recognition system 10 is mounted on a smartphoneas an example and iris matching is performed for identity verificationin a situation of user login, however, the iris recognition system 10 isnot limited thereto.

The iris recognition system 10 has a central processing unit (CPU) 101,a random access memory (RAM) 102, a read only memory (ROM) 103, and aflash memory 104 in order to implement a function as a computerperforming calculation and storage. Further, the iris recognition system10 has a communication interface (I/F) 105, a display device 106, aninput device 107, a visible light irradiation device 108, a visiblelight camera 109, an infrared irradiation device 110, and an infraredcamera 111. The CPU 101, the RAM 102, the ROM 103, the flash memory 104,the communication I/F 105, the display device 106, the input device 107,the visible light irradiation device 108, the visible light camera 109,the infrared irradiation device 110, and the infrared camera 111 areconnected to each other via a bus 112. Note that the display device 106,the input device 107, the visible light irradiation device 108, thevisible light camera 109, the infrared irradiation device 110, and theinfrared camera 111 may be connected to the bus 112 via a drive device(not illustrated) used for driving these devices.

While these components forming the iris recognition system 10 areillustrated as an integrated device in FIG. 1 , some of these functionsmay be formed by an externally attached device. For example, the visiblelight irradiation device 108, the visible light camera 109, the infraredirradiation device 110, or the infrared camera 111 may be an externallyattached device which is different from a part forming the function ofthe computer including the CPU 101 and the like.

The CPU 101 has a function of performing a predetermined operationaccording to a program stored in the ROM 103, the flash memory 104, orthe like and controlling respective components of the iris recognitionsystem 10. The RAM 102 is formed of a volatile storage medium andprovides a temporary memory area required for the operation of the CPU101. The ROM 103 is formed of a non-volatile storage medium and storesnecessary information such as a program used for the operation of theiris recognition system 10. The flash memory 104 is a storage devicethat is formed of a non-volatile storage medium and stores an imagecaptured by the infrared camera 111 or the like, an image of arecognition subject, feature amount data, or the like.

The communication I/F 105 is a communication interface in accordancewith a standard such as Wi-Fi (registered trademark), 4G, or the likeand also is a module for communicating with other devices. The displaydevice 106 is a liquid crystal display, an organic light emitting diode(OLED) display, or the like and used for displaying a moving image, astatic image, a text, or the like. The input device 107 is a button, atouch panel, or the like and is used for user operation of the irisrecognition system 10. The display device 106 and the input device 107may be integrally formed as a touchscreen.

The visible light irradiation device 108 and the visible light camera109 are provided on a display surface or the like of the display device106, for example. The visible light irradiation device 108 is used as alight source used for capturing by the visible light camera 109, forexample. Further, in a part of the process, the visible lightirradiation device 108 may be used as a visible light irradiation unitwhich performs a process of irradiating the user's eye with a visiblelight to induce a pupillary light reflex and thereby causing the pupilto contract. The visible light camera 109 can capture a landscape, auser's face, a user's eye, or the like by using a visible light toacquire an image. A digital camera with a Complementary Metal OxideSemiconductor (CMOS) image sensor, a Charge Coupled Device (CCD) imagesensor, or the like may be used for the visible light camera 109 so asto be suitable for image processing after capturing.

The infrared irradiation device 110 is a light emitting element such asan infrared LED that emits an infrared light. A digital camera with aCMOS image sensor, a CCD image sensor, or the like having a lightreceiving element configured to have a sensitivity to an infrared raymay be used for the infrared camera 111. By irradiating the user's eyewith an infrared light from the infrared irradiation device 110 andcapturing an infrared ray reflected from an iris by using the infraredcamera 111, it is possible to capture an iris image used for irismatching. By acquiring an iris image by using an infrared ray, it ispossible to obtain a high contrast image regardless of the color of theiris and reduce influence of reflection at a cornea. When such influencedoes not matter, however, the visible light camera 109 may be used tocapture an iris image. In this case, the infrared irradiation device 110and the infrared camera 111 may be omitted. Further, when the visiblelight camera is a camera capable of acquiring an image by using aninfrared ray in addition to a visible light, the infrared camera 111 maybe omitted. Note that the wavelength of the infrared ray emitted fromthe infrared irradiation device 110 may be a near infrared region around800 nm, for example.

Note that the hardware configuration illustrated in FIG. 1 is anexample, and other devices than the above may be added or some of theabove devices may not be provided. Further, some of the above devicesmay be replaced with another device having the same function.Furthermore, some of the functions may be provided by another device viaa network or may be implemented by the function forming the presentexample embodiment being distributed to a plurality of devices. Forexample, the flash memory 104 may be replaced with a hard disk drive(HDD) or may be replaced with cloud storage.

FIG. 2 is a function block diagram of the iris recognition system 10according to the first example embodiment. The iris recognition system10 has an image acquisition unit 121, a pupil detection unit 122, aniris detection unit 123, a determination object image extraction unit124, a size change unit 125, an evaluation value calculation unit 126, adetermination unit 127, a processing unit 128, a display unit 129, anextraction range setting unit 130, a feature amount calculation unit131, a matching unit 132, and a storage unit 133.

The CPU 101 implements the function of the image acquisition unit 121that acquires an infrared image of a user's eye by controlling theinfrared irradiation device 110 and the infrared camera 111. The CPU 101implements the function of the pupil detection unit 122, the irisdetection unit 123, the determination object image extraction unit 124,the size change unit 125, the evaluation value calculation unit 126, thedetermination unit 127, the processing unit 128, the extraction rangesetting unit 130, the feature amount calculation unit 131, and thematching unit 132 by loading a program stored in the ROM 103 or the liketo the RAM 102 and executing it. The process performed by each of theseunits will be described later. The display unit 129 displays theacquired image, a message to the user, or the like. The CPU 101implements the function of the display unit 129 by controlling thedisplay device 106. The storage unit 133 stores data of an iris imageacquired by the image acquisition unit 121, iris images registered inadvance, feature amounts calculated from these iris images, or the like.The CPU 101 implements the function of the storage unit 133 bycontrolling the flash memory 104.

FIG. 3 is a flowchart illustrating summary of the process performed bythe iris recognition system 10 according to the first exampleembodiment. In step S100, based on the image including the user's eye,the iris recognition system 10 determines whether or not the user's eyewears a colored contact lens.

A colored contact lens is a contact lens in which a colored part isprovided to a part of a lens. A colored contact lens may be referred toas a color contact lens, a circle contact lens, or a cosmetic contactlens. A colored contact lens worn on an eye can provide a decorativeeffect that changes the color of an iris or makes an iris appear larger.In many cases, a colored region of a colored contact lens is mainlyprovided to a portion corresponding to a part near the outercircumference of the iris, and a portion corresponding to the pupil iscolorless. This is for not losing a field of view while obtaining adecorative effect that makes an iris appear larger.

When a user wears a colored contact lens, a change in the pattern or thelike of an iris may affect the accuracy of iris matching. Usually,registration of an iris image is performed without a colored contactlens being worn. This is because, when matching is performed with acolored contact lens being worn, the pattern of the iris at registrationand the pattern of the iris at the matching are not matched. Thus, instep S100, prior to the iris matching, determination as to whether ornot a colored contact lens is worn is performed based on an imageincluding the user's eye. The content of the subsequent process can bemade more suitable based on whether or not a colored contact lens isworn.

In step S200, the iris recognition system 10 performs a process forimproving matching accuracy. This process is performed for reducingdeterioration in accuracy of iris matching due to the colored contactlens if it is determined that a colored contact lens is worn mainly instep S100.

In step S300, the iris recognition system 10 matches an image includingthe user's iris captured at the matching against an image including auser's iris registered in advance. In the matching process in this step,the content of the process can be changed in accordance with the resultof determination in step S100 as to whether or not a colored contactlens is worn. If a colored contact lens is worn, the iris recognitionsystem 10 performs matching of the iris by using a feature amountextracted from a region which excludes a predetermined range includingthe outer circumference of the iris out of the region of the irisincluded in the image.

It is not essential to perform all the processes from step S100 to stepS300 illustrated in FIG. 3 , and an example embodiment in which a partof the above processes is omitted is included in the scope of thepresent invention. For example, the matching accuracy improvementprocess of step S200 may be omitted to simplify the process. Further,the process of step S100 may be omitted to simplify the process bydetermining in advance whether or not a colored contact lens is worn byusing another device or instructing the user to input whether or notwearing a colored contact lens. Further, an image processing system thatalone performs a process of determining whether or not a colored contactlens is worn in step S100 is also included in the scope of the presentinvention. Further, the process from step S100 to step S300 may beperformed in cooperation of a plurality of devices.

With reference to FIG. 4 to FIG. 7 , a specific example of a coloredcontact lens-wearing determination process in step S100 of FIG. 3 willbe described. FIG. 4 is a flowchart illustrating a colored contactlens-wearing determination process according to the first exampleembodiment. The process of FIG. 4 is to more specifically represent theprocess in step S100 of FIG. 3 .

In step S101, the image acquisition unit 121 acquires an image includingthe user's eye. The acquired image is stored in the storage unit 133.Typically, because this image is acquired by an infrared ray, it is animage in grayscale. Further, the size of an image is 640 pixels in thehorizontal direction by 480 pixels in the vertical direction, forexample. However, since the size may be changed by a process describedlater, other sizes may be employed. Note that, in the presentspecification, terms expressing size such as the size of an image, thelength of a subject, or the like may be defined in a standardizednon-dimensional amount such as the number of pixels and are not limitedto those defined in a unit having the length dimension such asmile-meter. Further, the image including the user's eye is not limitedto that acquired by using an infrared ray and may be acquired by using avisible light.

In step S102, the pupil detection unit 122 detects a pupil from an imageincluding the eye acquired in step S101 and determines the position ofthe pupil. The determined position of the pupil is stored in the storageunit 133. The shape of the pupil can be approximated as a circle. Thus,the position of the pupil can be expressed by the center coordinates andthe radius of the pupil, for example. Note that the region of the pupilcan be detected by extracting pixels having lower brightness than apredetermined value, for example.

In step S103, the iris detection unit 123 detects an iris from the imageincluding the eye acquired in step S101 and determines the position ofthe iris. The position of the determined iris is stored in the storageunit 133. The pupil detection unit 122 and the iris detection unit 123may be more generally referred to as a detection unit. The shape of aniris can be approximated as an annular shape including the pupil, andthus the position of the iris can be expressed by the centercoordinates, the outer circumference radius, and the inner circumferenceradius of the iris, for example. Since the inner circumference radius ofan iris corresponds to the radius of the pupil, it can be omitted fromthe information indicating the position of the iris. Note that it ispossible to detect an iris by extracting a change in brightness at aboundary between the outer circumference of the iris and a sclera (socalled white of eye), for example. The iris detected here includes notonly the user's iris itself but also a pattern that is provided on acolored contact lens and represents an iris. Thus, in the presentspecification, the entire region appearing as an iris that includes aniris and a pattern provided on a colored contact lens may be referred toas an iris. Note that the order of step S102 and step S103 may beopposite. Alternatively, step S102 and step S103 may be performed inparallel or may be performed as a single process.

In step S104, the determination object image extraction unit 124extracts two determination object images including side parts of an irisfrom an image including the eye acquired in step S101. The side parts ofan iris here refer to portions on the outer circumference of the iristhat are on the boundary to the white of eye, which are mainly the leftand right parts excluding the upper and lower parts hidden behind theeyelids. The extracted determination object images are stored in thestorage unit 133. These two determination object images are used for thecolored contact lens-wearing determination. Extraction of determinationimages will be described in detail with reference to FIG. 5 .

FIG. 5 is a diagram schematically illustrating extraction ofdetermination object images according to the present example embodiment.FIG. 5 schematically illustrates an eye 90, a pupil 91, and an iris 92included in the image acquired in step S101. The determination objectimage extraction unit 124 extracts two determination object images R1and R2 from the image of the eye 90. Note that the determination objectimage extraction unit 124 may be simply referred to as an extractionunit.

As illustrated in FIG. 5 , the determination object images R1 and R2 areextracted so as to include the left side part and the right side part ofthe iris 92, respectively. Each of the determination object images R1and R2 is a square whose side has a length of S (pixel). Further, oneside in the vertical direction with respect to the eye 90 of the sidesof the squares is a line segment passing through the center of the iris92. When the outer circumference radius of an iris is Ir1 (pixel), thelength S of one side is set to meet S=Ir1+m. Here, the value m is apredetermined value set so that the determination object images R1 andR2 include the side parts of an iris and include neither the upper endnor the lower end of the iris 92 and is set to 30 pixels, for example.

The reason why the positions of the determination object images R1 andR2 are set in such a way will be described. A pattern of a coloredcontact lens often has a notable feature near the outer circumference ofan iris. For example, an eye wearing a colored contact lens often has alarge change in brightness between the part of the iris and the part ofwhite of eye. Thus, inclusion of the outer circumference of an iris inthe determination object images R1 and R2 for determination improves theaccuracy in detecting a colored contact lens by using a determinationdevice. Further, the upper and lower parts of an iris may be hiddenbehind eyelids and eyelashes. When an eyelid or an eyelash is includedin the determination object images R1 and R2, the determination accuracymay deteriorate. It is therefore desirable to set the positions of thedetermination object images R1 and R2 so as to include the side parts ofan iris and include neither the upper end nor the lower end of the iris.Further, it is desirable that one of the sides in the vertical directionof each of the squares be a line segment passing through the center ofthe iris 92 because wide determination object images R1 and R2 can beensured.

Note that it is not essential that each of the determination objectimages R1 and R2 is a square. For example, each of the determinationobject images R1 and R2 may be a rectangle. That is, each shape of thedetermination object images R1 and R2 may be any shape that is notlimited to a square or a rectangle and may be a polygon, a sector, or afigure including a curve as an example of other shapes. When thedetermination object images R1 and R2 have shapes other than a square,it is desirable that the length in the horizontal direction of each ofthe determination object images R1 and R2 with respect to the eye 90 belonger than the outer circumference radius of the iris 92 so as toinclude the side parts of the iris and include the iris as widely aspossible.

While the determination object images R1 and R2 are acquired from theleft and right parts of the iris 92, any one of them may be acquired.However, it is desirable to use the two determination object images R1and R2 because this enables more accurate determination.

In step S105, the size change unit 125 changes the size of thedetermination object images R1 and R2. FIG. 6 is a diagram schematicallyillustrating a size change of the determination object image accordingto the first example embodiment. As illustrated in FIG. 6 , the sizechange unit 125 reduces the determination object images R1 and R2 havinga length of the side of S pixels to determination object images R1 a andR2 a having a length of the side of 32 pixels and thereby generates thedetermination object images R1 a and R2 a. The reason for changing thesize is to set images to have a certain number of pixels so as to besuitable for the subsequent determination process and thereby make aprocess performed by a determination device described later moreefficient. The determination object images R1 a and R2 a after the sizechange are stored in the storage unit 133. The length of one side being32 pixels is an example, and the invention is not limited thereto inparticular. Further, when it is not essential to limit the size in theprocess performed by a determination device, the process of step S105may be omitted.

FIG. 7 is a diagram schematically illustrating the determination processin step S106 to step S109. The subsequent process will be described withcross reference to FIG. 4 and FIG. 7 . The evaluation value calculationunit 126 that performs this process includes a determination deviceformed of a convolution neural network (CNN), for example. Many imagesincluding images of an eye wearing a colored contact lens and images ofan eye not wearing a colored contact lens are automatically learned inadvance as teacher data in the determination device, and thereby thedetermination device is configured to be able to determine whether ornot a colored contact lens is worn. Once an image including an eye isinput to the determination device, it outputs an evaluation valueindicating a likelihood of a colored contact lens being worn. Anevaluation value takes continuous values from 0 to 1 as an example. Inthis case, a higher likelihood of no colored contact lens being wornresults in an evaluation value closer to 0, and a higher likelihood of acolored contact lens being worn results in an evaluation value closer to1.

In step S106, a determination device of the evaluation value calculationunit 126 calculates two evaluation values V1 and V2 based on the twodetermination object images R1 a and R2 a, respectively. The evaluationvalue is stored in the storage unit 133.

From step S107 to step S109, the determination unit 127 determineswhether or not a colored contact lens is worn based on the twoevaluation values V1 and V2. More specifically, in step S107, thedetermination unit 127 determines whether or not both the evaluationvalue V1 and the evaluation value V2 are greater than or equal to 0.9.If both the evaluation value V1 and the evaluation value V2 are greaterthan or equal to 0.9 (step S107, YES), the determination unit 127determines that a colored contact lens is worn (step S108). If at leastone of the evaluation value V1 and the evaluation value V2 is less than0.9 (step S107, NO), the determination unit 127 determines that nocolored contact lens is worn (step S109).

As discussed above, in the determination process from step S106 to stepS109, only when both the determination results from the twodetermination object images R1 a and R2 a indicate that a coloredcontact lens is worn, it is determined that a colored contact lens isworn. There are various types of colored contact lenses, and there is anindividual difference in the pattern of the iris or the like. Thus, evenwhen a sufficient amount of teacher data is used to perform learning,the determination device may make erroneous determination. Accordingly,in order to improve determination accuracy, it is desirable to inputboth the two determination object images R1 a and R2 a as describedabove and aggregate determination results from them to makedetermination.

An example of a process when the determination process from step S106 tostep S109 is performed by using a single determination object image willbe described. The determination device of the evaluation valuecalculation unit 126 calculates a single evaluation value from thesingle determination object image. Then, the determination unit 127determines that a colored contact lens is worn if the evaluation valueis greater than or equal to 0.9 and determines that no colored contactlens is worn if the evaluation value is less than 0.9, which enablesdetermination as to whether or not a colored contact lens is worn. Inthis way, the number of determination object images may be one.

As discussed above, according to the colored contact lens-wearingdetermination process of the present example embodiment, by performingthe determination based on a determination object image including theside part of the outer circumference of an iris, it is possible toaccurately determine whether or not a colored contact lens is worn.

Note that, while the determination device with a convolution neuralnetwork is used in the determination process illustrated in FIG. 7 ,without being limited thereto, determination as to whether or not acolored contact lens is worn may be performed based on a change inbrightness at the boundary between the outer circumference of the irisand the white of eye, for example.

Further, the determination in step S107 may be a process of determiningthat a colored contact lens is worn if at least one of the evaluationvalue V1 and the evaluation value V2 is greater than or equal to 0.9 anddetermining that no colored contact lens is worn if both of theevaluation value V1 and the evaluation value V2 are less than 0.9.Further, the determination in step S107 may be a process of selectingone or more evaluation values to be used in the determination out of aplurality of evaluation values such as the evaluation value V1, theevaluation value V2, and the like in accordance with the reliability orthe like of the determination object image. For example, wheninsufficiently reliable one is included in a plurality of determinationobject images due to noise, it is possible to improve the determinationaccuracy by excluding an evaluation value of such a determination objectimage from the determination process. Note that, in the exampledescribed above, the threshold of 0.9 used for determination of theevaluation value V1 and the evaluation value V2 is a mere example andmay be set to any value other than 0.9.

Next, a specific example of the matching accuracy improvement process instep S200 of FIG. 3 will be described with reference to FIG. 8 to FIG.10 .

FIG. 8 is a flowchart illustrating a first example of the matchingaccuracy improvement process according to the first example embodiment.In step S201, the processing unit 128 refers to the determination resultof step S100. If it is determined that the user wears a colored contactlens (step S201, YES), the process transfers to step S202. If it isdetermined that the user does not wear a colored contact lens (stepS201, NO), the process ends.

In step S202, the processing unit 128 controls the visible lightirradiation device 108 to irradiate the user' eye with a visible light.Then, in step S203, the processing unit 128 causes the image acquisitionunit 121 to again acquire an image including the eye.

When a retina of the eye is irradiated with a visible light, a pupillarylight reflex occurs that causes a pupil to contract. This results in awider iris region and a wider area in which the pattern of the iris canbe acquired. Therefore, deterioration in accuracy of iris matching dueto a colored contact lens can be reduced. Further, in the presentexample, since an unconscious reflex is utilized, the recognitionsubject is not required to take any action. For example, the presentexample can be applied to a case where it is difficult to make therecognition subject to take some action, such as when the recognitionsubject is not cooperative in iris matching or when the recognitionsubject does not understand the language, such as in a situation ofidentity verification in a government office.

Even when either one of the eyes is irradiated with a visible light,contraction of the pupil due to a pupillary light reflex occurs in boththe eyes. Thus, the eye irradiated with a visible light and the eye usedfor again acquiring an image may be different. This can suppress theinfluence of a reflex caused by the irradiated visible light.

FIG. 9 is a flowchart illustrating a second example of the matchingaccuracy improvement process according to the first example embodiment.The description of the same process as that in FIG. 8 will be omitted asappropriate. In step S204, the processing unit 128 causes the displayunit 129 to display a message that prompts the user to look at a brightobject. An example of a specific message may be “To authenticate oncemore, please look at something bright and then look at camera again.”After the user looks at a bright object in response to this message, instep S203, the processing unit 128 causes the image acquisition unit 121to again acquire an image including the eye.

Also in the present example, because the user looks at a bright object,a pupillary light reflex causes the pupil to contract and the region ofthe iris to be wider. It is therefore possible to reduce deteriorationin accuracy of iris matching due to a colored contact lens. In thepresent example, unlike the first example described above, it ispossible to perform the matching accuracy improvement process utilizinga pupillary light reflex even when the visible light irradiation device108 is not provided. Note that the message may have a content thatspecifically indicates a thing to look at such as “To authenticate oncemore, please once look at lighting and then look at camera again.”

An example in which the matching accuracy is significantly improved inthe configuration utilizing contraction of the pupil in FIG. 8 and FIG.9 will be described. If it is determined that a colored contact lens isworn in step S100, a process of excluding a part near the outercircumference of the iris from the extraction range of the featureamount may be performed in the iris matching in step S300. At this time,there may be a problem of decrease in the region contributing toextraction of a feature amount and thus a reduction in the accuracy ofiris matching. In such a situation, by performing the process of FIG. 8and FIG. 9 to cause the pupil to contract, it is possible to increasenot the area near the outer circumference of the iris but the oppositeside, namely, the area of the inner circumference side of the iris. Thisenables accurate iris matching even when a part near the outercircumference of the iris is excluded from the extraction range of afeature amount.

FIG. 10 is a flowchart illustrating a third example of the matchingaccuracy improvement process according to the first example embodiment.The description of same process as that in FIG. 8 will be omitted asappropriate. In step S205, the processing unit 128 causes the displayunit 129 to display a message that prompts the user to blink. A specificexample of the message may be “To authenticate once more, please blinkand then look at camera again.” After the user blinks in response tothis message, in step S206, the processing unit 128 causes the imageacquisition unit 121 to additionally acquire an image including the eye.

In the present example, the user is caused to blink and thereby thecolored contact lens rotates, and the position of the pattern of thecolored contact lens changes before and after the blinking. On the otherhand, the pattern of the iris itself of the user's eye does not changeby a blinking. Therefore, by calculating a difference between two imagesacquired before and after a blinking, it is possible to separate thepattern of the colored contact lens from the pattern of the iris of theuser's eye and thus perform accurate matching by extracting the patternof the iris of the user's eye. It is therefore possible to reducedeterioration in accuracy of iris matching due to a colored contactlens.

Also in the present example, the accuracy improvement process can beperformed. In the present example, since a pupillary light reflex causedby a change in brightness is not utilized, there is an advantage of theeffect being not lost even in a bright place.

Note that the process in the flowchart of FIG. 10 may be modified into aprocess of repeating the process of step S206 for multiple times toacquire images for multiple times. Furthermore, in this case, theprocess of step S205 may be omitted. Since a human blinks unconsciously,such an unconscious blinking may cause rotation of the colored contactlens even without display of a message that prompts the user to blink.Therefore, even without displaying a message, repeated acquisition ofimages may enable acquisition of images used for separating the patternof the colored contact lens and the pattern of the iris of the user'seye from each other, and in this case, the same effect and advantage asthat described above can be obtained.

FIG. 11 is a flowchart illustrating a modified example of the thirdexample of the matching accuracy improvement process according to thefirst example embodiment. As illustrated in step S207 of FIG. 11 , thecontent of the message may be to prompt the user to move the eye such asan instruction of looking right and left. A specific example of themessage may be “To authenticate once more, please look right and leftand then look at camera again.” Since motion of the eyeball causesrotation of the colored contact lens, the same effect and advantage asthat of the case of prompting a blinking as described above can beobtained also in the present modified example.

As discussed above, according to the matching accuracy improvementprocess of the present example embodiment, it is possible to reducedeterioration in accuracy of iris matching due to a colored contact lensbeing worn.

Note that, while the user is notified of various messages by display onthe display unit 129 in the processes described above, a method ofnotification of the message is not limited thereto. For example, voicemay be used for notification of the message.

A specific example of the iris matching process in step S300 of FIG. 3will be described with reference to FIG. 12 and FIG. 13 . FIG. 12 is aflowchart illustrating the iris matching process according to the firstexample embodiment. The process of FIG. 12 is more specific version ofthe process of step S300 of FIG. 3 .

In step S301, the extraction range setting unit 130 refers to thedetermination result of step S100. If it is determined that the userwears a colored contact lens (step S301, YES), the process transfers tostep S302. If it is determined that the user does not wear a coloredcontact lens (step S301, NO), the process transfers to step S303.

In step S302, the extraction range setting unit 130 excludes a part nearthe outer circumference of the iris to set an extraction range of afeature amount. FIG. 13 is a diagram schematically illustrating asetting of the extraction range of the feature amount according to thefirst example embodiment. As illustrated in FIG. 13 , the region of theiris in an image is divided into a mask region 93 and a feature amountextraction region 94. The mask region 93 is a region defined inaccordance with a position of the outer circumference and the innercircumference of the iris, which is, out of the region of the iris, arange from which no feature amount used for the subsequent matchingprocess is extracted. The feature amount extraction region 94 is, out ofthe region of the iris, a region from which a feature amount isextracted and which is a range except the mask region 93. The maskregion 93 is arranged near the outer circumference of the iris. That is,the feature amount extraction region 94 is a range excluding apredetermined range including the outer circumference of the iris.

The range of the mask region 93 is defined as a ring including the outercircumference end of the iris and having a width of 60 percent or morefrom the outer circumference to the inner circumference of the iris.More specifically, the mask region 93 is a ring having an outercircumference radius is Ir1 and an inner circumference radius is{Ir2+0.4×(Ir1−Ir2)}, where the outer circumference radius of the iris isIr1 and the inner circumference radius of the iris is Ir2.

The reason for defining the mask region 93 and the feature amountextraction region 94 will be described. In a colored contact lens, manycolored portions (pattern) are arranged in a portion corresponding to apart near the outer circumference of an iris. Thus, when a featureamount is extracted from a region including a part near the outercircumference of the iris to perform iris authentication with a coloredcontact lens being worn, iris matching is performed with the pattern ofthe colored contact lens overlapping with the pattern of the iris, andtherefore this causes a problem of a reduction in matching accuracy. Forexample, the same person may be determined as another person because thepattern of the iris not wearing a colored contact lens taken atregistration does not match the pattern taken at the matching with thepattern of the colored contact lens overlapping with the pattern of theiris.

By defining the feature amount extraction region 94 as a range excludinga predetermined range including the outer circumference of the iris, apart near the outer circumference of the iris in which many patterns ofthe colored contact lens are arranged is excluded from the extraction ofa feature amount. On the other hand, no pattern may be arranged in aregion of the colored contact lens corresponding to a part near theinner circumference of the iris in many cases in order to ensure a fieldof view, which less affects iris matching. It is thus possible to reducedeterioration in accuracy of iris matching due to a colored contact lensbeing worn.

In step S303 when the user does not wear a colored contact lens, themask region 93 is not provided, and the feature amount extraction region94 is set for the entire iris. When no colored contact lens is worn, itis not necessary to exclude a part near the outer circumference from thefeature amount extraction region, rather, no exclusion of a part nearthe outer circumference allows for a wider feature amount extractionregion and thus enables higher matching accuracy. It is thereforedesirable to perform different process depending on whether or not acolored contact lens is worn.

In step S304, the feature amount calculation unit 131 uses the patternsof the iris included in an image of the set feature amount extractionregion 94 or the like to calculate a feature amount. The calculatedfeature amount is stored in the storage unit 133.

In step S305, the matching unit 132 matches the feature amount acquiredin step S304 against a feature amount of an iris of a user registered inadvance in the storage unit 133. When it is determined as the iris ofthe same person by matching, the iris recognition system 10 determinesthat the identity verification is completed and performs a process oflogin or the like.

As discussed above, according to the iris matching process of thepresent example embodiment, by defining a range excluding apredetermined range including the outer circumference of the iris as anextraction range of a feature amount, it is possible to reduce influenceon matching accuracy when the recognition subject wears a coloredcontact lens.

When excluding a part near the outer circumference of the iris to setthe feature amount extraction region 94 by using the scheme of stepS302, it is more effective to combine any of the matching accuracyimprovement processes illustrated by using FIG. 8 and FIG. 9 . In thescheme of step S302, the feature amount extraction region 94 is reducedby the area of the mask region 93. Using the process of FIG. 8 or FIG. 9together, it is possible to increase the range of the iris and thereforecompensate the influence of the reduced feature amount extraction region94.

In the process described above, since the mask region 93 is determinedindependently of the pattern of a colored contact lens, there is no needto perform a process of detecting the pattern of the colored contactlens, which enables a high speed process. When the accuracy is moreimportant than the processing speed, however, a process of determining acolored portion of a colored contact lens by determining the pattern ofthe colored contact lens and the pattern of an iris itself of the usermay be added. In this case, the colored portion of the colored contactlens is set as the mask region 93. In such a way, when the coloredportion of a colored contact lens (the shape of the pattern or theregion where the pattern is arranged) can be determined, the mask region93 that is a region near the outer circumference of the iris when irismatching is performed can be set based on the colored portion of thecolored contact lens.

As an example of the process of determining the pattern of a coloredcontact lens and the pattern of an iris itself of the user, there may bea scheme of searching for the contour of the pattern of the coloredcontact lens by using an adaptive contour model such as a snakes methodor a Level Set method. Further, as another example, there may be ascheme of dividing an iris into a plurality of rectangular regions andusing a convolutional neural network for each rectangular region todetermine whether or not there is a pattern of a colored contact lens, ascheme of detecting whether or not there are regular figures on aconcentric circle, a scheme of detecting whether or not there are radialstraight lines, or the like.

The device described in the above example embodiment can be alsoconfigured as a second example embodiment to a fourth example embodimentillustrated below.

Second Example Embodiment

FIG. 14 is a function block diagram of an iris recognition system 20according to the second example embodiment. The iris recognition system20 has a determination unit 201 and a matching unit 202. Thedetermination unit 201 determines whether or not a colored contact lensis worn based on an image including an eye of a recognition subject. Thematching unit 202 performs matching of the iris by using a featureamount extracted from a region excluding a predetermined range includingthe outer circumference of the iris out of the region of the irisincluded in the image when it is determined by the determination unit201 that a colored contact lens is worn.

According to the present example embodiment, the iris recognition system20 that can reduce influence on the matching accuracy when therecognition subject wears a colored contact lens can be provided.

Third Example Embodiment

FIG. 15 is a function block diagram of an image processing system 30according to the third example embodiment. The image processing system30 has an extraction unit 301 and the determination unit 302. Theextraction unit 301 extracts a determination object image including aside part of the outer circumference of an iris from an image includingan eye. The determination unit 302 determine whether or not a coloredcontact lens is worn based on the determination object image.

According to the present example embodiment, the image processing system30 that can accurately determine whether or not a colored contact lensis worn can be provided.

Fourth Example Embodiment

FIG. 16 is a function block diagram of an iris recognition system 40according to the fourth example embodiment. The iris recognition system40 has a determination unit 401 and the processing unit 402. Thedetermination unit 401 determines whether or not a colored contact lensis worn based on an image including an eye of a recognition subject. Theprocessing unit 402 performs a process of improving accuracy of irismatching on the recognition subject when it is determined by thedetermination unit 401 that a colored contact lens is worn.

According to the present example embodiment, the iris recognition system40 that can reduce deterioration in accuracy of iris matching due to acolored contact lens being worn can be provided.

Modified Example Embodiments

The present invention is not limited to the example embodimentsdescribed above and can be changed as appropriate within the scope notdeparting from the spirit of the present invention.

The iris recognition systems of the example embodiments described abovemay be used not only for authentication at login illustrated as anexample but also for various identity verification. For example, theiris recognition systems of the example embodiments described above maybe applied to identity verification for immigration at an airport, aseaport, or a national border, identity verification at a governmentoffice, identity verification for access to a factory or an office,identity verification at entrance to an event hall, or the like.

In each example embodiment described above, acquisition of an image usedfor iris matching may be performed on one eye or may be performed onboth eyes of an authentication subject. Acquisition of an image of onlyone eye has the advantage of improvement of the processing speed orreduction in the storage capacity, and acquisition of images of botheyes has the advantage of improvement of authentication accuracy.

In each example embodiment described above, a situation where a featureamount of an acquired image of an iris is matched against a featureamount of an iris which has been registered in advance, that is, aprocess of iris matching has been described as an example. However, eachexample embodiment described above can be applied to a situation ofregistration of an image of an iris or a feature amount of an iris withappropriate modification. Specifically, with the iris matching processof step S300 being replaced with the iris registration process,modification can be made for the situation of registration. In thiscase, the iris registration process may be a process of acquiring animage of an iris, calculating a feature amount, and storing the featureamount of the iris, for example.

While the process of step S202, S204, or S205 is performed when it isdetermined that the user wears a colored contact lens in the process ofstep S201 in each example embodiment described above, this process maybe other process than is illustrated. For example, it may be a processthat, when it is determined that the user wears a colored contact lens,displays a message instructing the user to put off the colored contactlens. Alternatively, when it is determined that the user wears a coloredcontact lens, a failure of matching or registration may be determined.In this case, the process may notify the user of a failure of theprocess by a message, an alarm, or the like to prompt another matchingor registration or may notify the user of a failure of the process toend the process. Note that these processes may be performed instead ofthe process of step S202, S204, or S205 or may be performed in parallelwith the process of step S202, S204, or S205.

As described above, the advantage of instructing the user to put off thecolored contact lens or performing the process as matching orregistration being failed will be described with an example of asituation of iris registration. Since the importance of identityverification is high in a situation of iris registration, it isnecessary to enhance reliability of a registered iris being the personin question. In particular, this requirement for reliability is moresignificant in a situation of important authentication such asimmigration control or national identification (ID). Registration with acolored contact lens being worn will impair the function of identityverification. Thus, when registration is attempted with a coloredcontact lens being worn, the reliability of identity verification can bemaintained by issuing some alert and disabling registration. Thisimproves the accuracy of an iris recognition system. Further, multipleregistration using a colored contact lens may be prevented.

The scope of each example embodiment also includes a processing methodof storing, in a storage medium, a program that causes the configurationof the example embodiment to operate to implement the function of theexample embodiment described above, reading out as a code the programstored in the storage medium, and executing the code in a computer. Thatis, a computer readable storage medium is included in the scope of eachexample embodiment. Further, not only the storage medium in which theprogram described above is stored but also the program itself isincluded in each example embodiment. Further, one or more componentsincluded in the example embodiments described above may be a circuitsuch as an Application Specific Integrated Circuit (ASIC), a FieldProgrammable Gate Array (FPGA), or the like configured to implement thefunction of each component.

As the storage medium, a floppy (registered trademark) disk, a harddisk, an optical disk, a magneto-optical disk, a Compact Disk (CD)-ROM,a magnetic tape, a non-volatile memory card, or a ROM, for example, maybe used. The scope of each example embodiment includes not only thoseexecuting a process with a program itself stored in the storage mediumbut also those operating on an operating system (OS) in cooperation withthe function of another software or an extension board to execute theprocess.

Services realized by the function of each example embodiment describedabove can be provided to the user in a form of Software as a Service(SaaS).

Note that all the example embodiments described above are mere examplesof embodiment in implementing the present invention, and the technicalscope of the present invention should not be limited to these exampleembodiments. That is, the present invention can be implemented invarious forms without departing from the technical concept thereof orthe primary feature thereof.

The whole or part of the example embodiments disclosed above can bedescribed as, but not limited to, the following supplementary notes.

(Supplementary Note 1)

An image processing system comprising:

an extraction unit that extracts a determination object image includinga side part of an outer circumference of an iris from an image includingan eye; and

a determination unit that determines whether or not a colored contactlens is worn based on the determination object image.

(Supplementary Note 2)

The image processing system according to supplementary note 1, whereinthe determination object image includes neither an upper end nor a lowerend of the outer circumference of the iris.

(Supplementary Note 3)

The image processing system according to supplementary note 1 or 2,wherein a length in a horizontal direction of the determination objectimage with respect to the eye is longer than an outer circumferenceradius of the iris.

(Supplementary Note 4)

The image processing system according to supplementary note 3,

wherein the determination object image is square, and

wherein a side of the square has a length obtained by adding apredetermined value to the outer circumference radius of the iris.

(Supplementary Note 5)

The image processing system according to supplementary note 4, whereinthe side of the square is a line segment passing through a center of theiris.

(Supplementary Note 6)

The image processing system according to any one of supplementary notes1 to 5 further comprising an evaluation value calculation unit thatincludes a determination device formed of a convolution neural networkand calculates, by inputting the determination object image into thedetermination device, an evaluation value indicating a likelihood of thecolored contact lens being worn.

(Supplementary Note 7)

The image processing system according to any one of supplementary notes1 to 5, wherein the determination unit determines whether or not thecolored contact lens is worn based on a change in brightness at aboundary between the outer circumference of the iris and a white of theeye.

(Supplementary Note 8)

The image processing system according to any one of supplementary notes1 to 7 further comprising a size change unit that changes a number ofpixels of the determination object image into a predetermined number ofpixels prior to the determination by the determination unit.

(Supplementary Note 9)

The image processing system according to any one of supplementary notes1 to 8, wherein the extraction unit extracts two determination objectimages including both sides of the outer circumference of the iris,respectively.

(Supplementary Note 10)

The image processing system according to any one of supplementary notes1 to 9, the determination unit determines that the colored contact lensis worn when both of two determination object images indicate that thecolored contact lens is worn.

(Supplementary Note 11)

The image processing system according to any one of supplementary notes1 to 9, the determination unit determines that the colored contact lensis worn when at least one of two determination object images indicatesthat the colored contact lens is worn.

(Supplementary Note 12)

An image processing method comprising:

extracting a determination object image including a side part of anouter circumference of an iris from an image including an eye; and

determining whether or not a colored contact lens is worn based on thedetermination object image.

(Supplementary Note 13)

A storage medium in which a program is stored, the program causing acomputer to execute:

extracting a determination object image including a side part of anouter circumference of an iris from an image including an eye; and

-   -   determining whether or not a colored contact lens is worn based        on the determination object image.

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2017-165404, filed on Aug. 30, 2017, thedisclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

-   10, 20, 40 iris recognition system-   30 image processing system-   90 eye-   91 pupil-   92 iris-   93 mask region-   94 feature amount extraction region-   121 image acquisition unit-   122 pupil detection unit-   123 iris detection unit-   124 determination object image extraction unit-   125 size change unit-   126 evaluation value calculation unit-   127, 201, 302, 401 determination unit-   128, 402 processing unit-   129 display unit-   130 extraction range setting unit-   131 feature amount calculation unit-   132, 202 matching unit-   133 storage unit-   301 extraction unit-   R1, R2, R1 a, R2 a determination object image

The invention claimed is:
 1. An image processing system comprising: amemory configured to store instructions; and a processor configured toexecute the instructions to: perform a machine learning by using animage of an eye wearing a colored contact lens, an image of an eye notwearing a colored contact lens, and information indicating whether acolored contact lens is worn as teacher data; and generate, based on aresult of the machine learning, a learned model that outputs informationindicating whether a colored contact lens is worn by inputting an imageof an eye wearing a colored contact lens or an image not wearing acolored contact lens.
 2. An image processing system comprising: a memoryconfigured to store instructions: and a processor configured to executethe instructions to: generate a learned model by performing a machinelearning by using an image of an eye wearing a colored contact lens, animage of an eye not wearing a colored contact lens, and informationindicating whether a colored contact lens is worn as teacher data; anddetermine, by using the learned model, whether a colored contact lens isworn by inputting an image of an eye wearing a colored contact lens oran image not wearing a colored contact lens.